1. Field of the Invention
The present invention relates to photography and, more particularly, to extracting objects from a composite scene being shot by a video camera. Extraction may be achieved by lighting a first object (such as a foreground object) in visible light and lighting a second object (such as a background object) in infrared and visible light. A visible light camera and an infrared light camera are used in the video shoot. The infrared light camera output is used as a key for object extraction. If the second object, for example, is not desired in the scene, the infrared light camera output is used as a key to remove the second object from the visible light camera output. If the second object is extracted, it may be replaced by another video or graphic image.
2. Discussion of Related Art
In video scenes in which multiple objects are being imaged, it is often advantageous to separate the image of a first object, such as a live person or other object from the scene. It is also desirable to accomplish this removal without altering the existing arrangement of objects in the scene.
One current object extraction practice, called "blue-screening", uses a monochromatic (often blue) background mounted behind the object (often a person) whose background is to be removed. The foreground objects are imaged against this monochromatic background by a video camera. The output of the video camera is then applied to a processing circuit which "keys" on the blue signal and removes it from the image, replacing it on a pixel by pixel basis with the video image from some other video signal. The final composite image appears to have the foreground objects in the original scene superimposed over the video from a completely different scene. One well known example of "blue-screening" is a TV weatherperson superimposed over a video view of a weather map.
One problem with "blue-screening" and other currently known color-keying extraction techniques, however, is that the scene needs a physical, monochromatic background. Thus, an "on-location" broadcast cannot use this extraction method unless a portable blue screen is available to place behind the foreground object (such as a newscaster). In addition, the background color cannot be present in the foreground object because the color is treated in the same manner as the background--it is removed. Thus, the blue-screen technique typically requires the foreground object, such as a newscaster, to be present in a studio or other specially prepared room. Also, the foreground object or any other portion of the foreground scene is restricted in usable color schemes (e.g., blue cannot appear). Television weather forecasters being blue-screened, for instance, cannot wear blue and must stand in front of a blue screen.
The problem is exacerbated in the case of extracting backgrounds behind participants in a video conference. For instance, in a multi-point video conferencing system such as may be accomplished using Bellcore's Proprietary Personal Presence System (described in U.S. patent Application Ser. Nos. 08/434,081 entitled "System and Method for Associating Multimedia Objects"; 08/434,083 entitled "Infinitely Expendable Real-Time Video Conferencing System" now U.S. Pat. No. 5,737,011, Apr. 7, 1996; and 08/432,242 entitled "Real-Time Video Conferencing System and Method with Multilayer Keying of Multiple Video Images" now U.S. Pat. No. 5,657,096, Aug. 12, 1997; all assigned to the assignee of this invention and all filed on May 3, 1995. The contents of these applications are incorporated herein by reference), multiple conferees engaged in a video conference can see the images of any, or all, of the other conferees on their viewing screens. Users may also customize their views of other conferees.
Video conferencing users may want the ability to remove the images of selected other conferees from their backgrounds. For instance, a user may wish to overlay just the conferees, without their backgrounds, on a common video background, thus providing the image that those conferees are all in the same room. A problem arises when conferee A wants to remove conferee B from his background, while conferee C wants to see conferee B's natural background. This is not possible with conventional blue-screen techniques. This is because, for example, if B appears in front of a blue screen to accommodate A, B's natural background is obscured by the blue screen, thus preventing C from seeing B's background.
A video conferencing user or picture phone user may desire to conceal the background from another conferee or caller for privacy reasons. Using conventional methods, the user must have a physical, monochrome background to conceal the background. This may be impractical, particularly for a picture phone located in a home.
Thus, it is an object of the present invention to permit a teleconference participant to choose freely how other conferees are viewed, including what background is displayed.
It is another object of the present invention to provide object extraction wherein the foreground object color scheme (such as dress and/or color content of persons or objects being video imaged) is unrestricted.
It is yet a further object of the present invention to provide object extraction that does not require the foreground object to be placed in front of a monochromatic background, thus permitting object extraction in "on-location" and other types of video shoots where it may be difficult or inconvenient to provide a physical, monochromatic background.